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ABSTRACT 



An active independent research and development (IR&D) 
program is a contributing factor to the U.S. military's 
reputation for technologically superior weapon systems and 
combat support equipment. This thesis examines the current 
selection process of IR&D projects at Naval Research, Develop- 
ment, Test & Evaluation (RDT&E) Centers and develops a 
recommendation to tailor the selection process to the charac- 
teristics of the project under consideration. The U.S. Navy 
divides its IR&D projects into two categories, independent 
research (IR) and independent exploratory development (IED) . 
This thesis recommends that a scoring method be used to select 
IR projects and an economic method be used to select IED 
projects. The thesis concludes by discussing future issues 
that will impact the IR&D programs. 
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I. 



INTRODUCTION 



The United States military has established a reputation 
for being on the cutting edge of technology regarding its 
weapon systems and combat support equipment. One contributing 
factor to this reputation is an active independent research 
and development (IR&D) program within private industry and 
government laboratories. 

As the pendulum of military funding swings back (this does 
not include the Persian Gulf crisis, which is an emergency 
funding situation) to fewer funds being available at all 
levels of military operations, the need to optimize funding of 
IR&D becomes more acute if the U.S. is to continue to upgrade 
its military technological base. Current funding plans show 
that Congress and the President are committed to maintaining 
research and development funding at the present level. [Ref. 
l:p. 198] One small portion of this budget is IR&D, which for 
this paper is defined as research to advance the state of the 
art, to solve problems of interest to the Department of 
Defense (DoD) , and to extend the capabilities of contractors 
and government laboratories to solve DoD problems [Ref. 2:p. 
126] . The process of establishing an optimal funding pattern 
in this area is not an easy task. The selection of IR&D 
projects may share some basic characteristics with other 
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selection and allocation decisions, but it is sufficiently 
unique to have generated its own body of research literature. 

To fully understand the difficulties involved in 
selecting IR&D projects it is necessary to understand those 
traits which tend to contribute to uncertainty in the 
selection process. First, final decision making tends to be 
relatively centralized, while essential information is spread 
throughout the organization leading to decision making without 
complete information. This situatic is aggravated by the 
typical multi-layered structure for gathering and processing 
information as numerous individuals with sign-off powers 
delete, change and add information prior to submission to 
final decision makers [Ref. 3:p. 1257]. 

Second, the benefits of IR&D tend to have multiple 
dimensions, some of which are not easy to translate into a 
cost figure. For example, it is difficult to establish a 
dollar value on the scientific reputation of an organization 
or the value of such a reputation for attracting top quality 
scientists and engineers. Another benefit that is difficult 
to define in terms of dollars is the importance of developing 
areas of expertise in selected areas, even if applications are 
several years in the future. [Ref. 4:p. 8] 

Third, the reliability of cost estimation models in this 
field vary tremendously. Reliable cost estimation models tend 
to be in fields where the research involves improvements on 
known technology or historical data exists for similar systems 
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[Ref. 5: p.13]. The reliability of cost estimation figures 
deteriorate rapidly as the research shifts into new 
technological areas. 

Finally, it is difficult to establish a quantitative 
relationship between the IR&D project and its potential for 
success because the term "potential” indicates a non- 
quantitative measurement. The term "success" also lends 
itself to ambiguity. Industry usually defines success as 
profitability, while government has several definitions: 
operability, reputation or national prestige, technical merit, 
etc . 

There are other minor characteristics that contribute to 
the high degree of uncertainty in the selection process of 
IR&D projects but it is clear from the ones discussed that 
optimizing project selection is difficult under these 
conditions. 

There exist several different methods or models in the 
literature to aid decision makers in the selection of IR&D 
projects. These models can be divided into four general 
categories for evaluation of IR&D projects: economic methods, 
decision theory methods, constrained optimization models and 
scoring. Each of these categories will be reviewed in depth 
in Chapter II. At this point a brief description will 
suffice. 

- Economic Methods: a group of models utilizing cost 
effectiveness methods to determine economic feasibility. 
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This method is most commonly used for product-oriented 
R&D by using standard capital budgeting techniques. 

Decision Theory Methods: models using simulation 
analysis in order to generate a range of outcomes to 
reflect a probability density function of expected value. 

Constrained Optimization Models: models using 
mathematical programing to optimize an objective 
function (s) subject to specified resource constraints. 

Scoring: methods requiring the identification and 
assignment of merit to each project with respect to a 
priori criteria. [Ref. 6:p. 21] 

Each of these methods contribute to the knowledge of how 
to optimize funding patterns in the area of selection of -^&D 
projects and its own set of advantages and disadvantages. A 
crucial decision facing government organizations is to 
determine which method meets the needs of their organization 
and how to implement that method. 

/ 

The Space and Naval Warfare Systems Command provides 
guidance to eight Naval Research, Development, Test and 
Evaluation (RDT&E) centers. Each of these centers manages an 
active IR&D program in ver diverse fields while operating 
under the same general management guidelines. Chapter III 
will discuss these guidelines and describe each center's 
mission, area of expertise and scope of their IR&D program. 
This chapter will illustrate the diversity and range of 
research being conducted at these eight centers and set the 
stage for the following chapter which recommends a more 
sophisticated method of selecting IR&D projects while allowing 
for the individuality of the differe centers. 
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The final chapter will examine three future issues that 
face the RDT&E centers that will impact on their IR&D 
programs. The issues to be discussed include: (1) the 
consolidation of management and support responsibilities, (2) 
the age of Navy laboratories, facilities, and equipment, and 
(3) the emerging trend that stresses solving short-term 
engineering problems rather than investigating future 
technologies . 
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II. LITERATURE REVIEW 



A . INTRODUCTION 

The number of models in this field presented over time are 
too numerous to cover individually. This chapter will focus 
on describing four general categories. The models covered 
will be simplified versions which serve as the backbones for 
many of the variations that have appeared in e literature. 
The purpose of the chapter is to provide the :eader with a 
basic understanding of selection methods and their advantages 
and disadvantages. 

B. SCORING METHODS 

The process involved in setting up a scoring model in 
order to select projects is comprised of three steps. Those 
three steps in chronological order are the selection of the 
scoring criteria, the assignment of weights to those criteria, 
and the determination of scores [Ref. /:p. 34]. Subjective 
decisions are made at each step. The impact of subjective 
decision-making will be discussed further in the next section. 

A weighted score will be calculated for each project using 
a relatively simple mathematical model that sums the weighted 
criteria. Once a score has been determined for each project, 
the projects are ranked according to scores. Generally, the 
project with the highest score is designated as the most 
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preferred project. Occasionally a scoring model will be 
established to reflect the opposite where the lowest score is 
the preferred project. 

As noted earlier, each step requires subjective decisions 
to be made. If the reliability of the subjective decisions 
can be improved then the model will be improved. In the area 
of subjective decision-making, researchers have shown that 
group consensus is superior to the single person approach. 
[Ref. 8:p. 125] The common practice is for a single decision- 
maker to determine criteria and weights and to use a group to 
assign scores. Several sources recommend that groups be used 
throughout the scoring process to dramatically improve the 
model [Refs. 6:p. 24; 7:p. 34; 9:p. 553]. The scoring model 
described here does recommend the use of groups. 

1. Selecting Scoring Criteria and Weights 

The two steps of selecting scoring criteria and 
weights are considered together because they are preparatory 
in nature and once selected are subject to reviews, but their 
selection is not repeated for every project. The selection of 
the panel deserves some discussion. Members should be 
recognized experts in their area, whether it's technical, 
managerial, or marketing. An attempt should be made to create 
a heterogeneous group, as studies have shown that 
heterogeneous groups produce a higher proportion of high 
acceptance solutions than homogeneous groups [Ref. 10:p. 326]. 
The size of the group should be limited to less than nine 
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members because of the need for interaction, and the 
effectiveness of group techniques dwindles rapidly within 
larger groups [Ref. 7:p. 34]. 

Members within the panel develop a list of relevant 
factors affecting product success. These factors are usually 
identified as falling into two categories: technical factors 
and market factors (need for the product) . Usually these 
lists are quite extensive with 30 to 40 different factors 
being identified. Ob' ..ously using 30 to 40 different decision 
criteria would be extremely cumbersome and time consuming. 
Factors are then grouped together according to similarity to 
develop a manageable number of decision criteria; usually six 
to ten decision criteria are determined. 

Once decision criteria are set, it becomes necessary 
to establish relative weights. The simplest method is to 
establish a scale with values ranging from 1 (least favorable) 
to 5 (most favorable) , which is applied to each criterion. A 
scale with at least five divisions is required to produce 
significant statistical differences [Ref. ll:p. 21]. A more 
meaningful method would be to assign a percentile weight to 
each criterion (all weights sum to 100 percent) which would be 
multiplied by the selected scale value. This permits the 
organization to more heavily weigh key decision criteria. 
Well-established, clear organizational goals will be essential 
in order to properly weigh decision criteria [Ref. 12 :p. 223]. 
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2 . Scoring the Projects 



The normal process after the selection of scoring 
criteria and their relative weights is to provide scoring 
sheets to a number of specialists, who make independent, 
personal subjective decisions on each project. The projects 
are individually rated according to their merit on a 1 to 100 
scale for each decision criterion. These scores are then 
tabulated for each project according to the following model: 



n 

SUB s = l DC |S W, 
i=l 

where : 

SUB, = Total weighted score for project "s", 

DC,, = Value of decision criterion "i" for project 
"s" , 

W, = Weighing for decision criterion "i", 

s = Number of projects, 

n = Number of criteria. [Ref. 6:p. 24] 



The rankings obtained from the comparison of the total 
weighted score for each project provides the organization with 
a collective subjective opinion of which projects are likely 
to succeed. 

Another method for scoring the projects is to use 
scoring sheets in conjunction with the Delphi technique. 
Individual scoring sheets are summarized and a histogram of 
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distribution of scores is plotted. This information is 
resubmitted to individual evaluators for review and they are 
requested to revise their scores if they substantially differ 
from the group results. Results are again summarized and 
scores should converge to within a narrow range. It may be 
necessary to repeat the process but a narrow range should be 
achieved with two or three tries. A final score will be 
assigned based on a narrow range of scores for each project. 
[Ref. 7 : p. 37] 

3 . Advantages and Disadvantages 

The main advantages of scoring methods are their 
relative simplicity, their support of a we 11 -structured 
decision process that is easily understood, and their ability 
to incorporate diverse and non-monetary criteria. They have 
proven to be useful in the initial stages of project 
evaluation. They are particularly well-suited for screening 
decisions and preliminary analyses. 

Their main deficiency is that subjective decision- 
making is the basis for all aspects of this method. In 
reality, scoring provides a summarization of opinions. The 
determination of the weight values is extremely difficult to 
obtain with any precision, and failure to properly weigh 
decision criteria can easily skew the model. Further 
constraints on the reliability of this model are the degree of 
knowledge and mental endurance of the evaluators. [Ref. 13 :p. 
153] That is why as more precise information becomes 
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available other methods, such as the economic or optimization 
models, are usually more effective. 

C. ECONOMIC METHODS 

In its simplest form, an economic model involves 
constructing an index of benefit/cost ratios. Capital 
budgeting techniques are used to assign financial or dollar 
values to the benefit and cost variables of the project. The 
purpose of assigning dollar values is to try to develop a 
universal, quantitative measurement of IR&D projects. A 
subjective value concerning the likelihood of technical 
success is also assigned to the project. The function of the 
model is to select those projects which are likely to be 
successful and provide the greatest benefit while staying 
within the overall budget constraint. The following is a 
relatively simple example of a benefit/cost ratio. 

I, = (B, x T,)/C, 

where: 

i = 1, 2, ..., n where n is the number of 

projects, 

I, = Index value of project i, 

B, = Estimated value of project i, 

T, = Estimated chance of technical success of 
project i, 

C, = Estimated cost of research of project i. [Ref. 

14 :p. 26] 
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The next step is to maximize the value of benefits 
obtained within the constraint of the overall research budget. 
One method is to define a variable, S, , that can take on the 
value of 0 or 1 depending on the value of I,. In the case 
examined in this thesis of government laboratories with no 
profit-making requirement, any value greater than one (a 
reasonably safe breakeven point) will result in the initial 
selection of the project for further consideration. The 
organization car elect ly value that reflects its risk anc 
profit requirements as its threshold value. 

S, = 0 if I, < 1 (project not selected) 

1 if I, > 1 (project selected) 



This step eliminates all projects that fail to meet the 
risk and profit requirements of the organization and 
establishes a group of qualified projects. Using the same 
variables identified in the benefit/cost ratio ^or defining 
estimated value, chance of technical success a:.d estimated 
cost; the selection process is now a maximization problem 
constrained by the overall IR&D budget. 

n 

maximize Z = £ B^S, 

i=l 

n 

subject to £ C,S, < C 
i=l 

where C = total cost. [Ref. 14 :p. 27] 
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There are several variations of this basic model. 



One 



popular technique, which is discussed below, is to incorporate 
discounted cash flow analysis into this model. 

1 . Discounted Cash Flows 

A slightly more sophisticated approach is to use the 
present value of benefit and cost variables either alone or 
with the above model. The reasoning behind this is that all 
money has a time value. As the focus of the economic model is 
to examine financial resources, present value provides a more 
realistic assessment of benefit and cost values. 

Discounted cash flow refers to the techniques used to 
calculate the time value of money. J. Hamaker defines cash 
flow as "the expected life cycle costs and revenues of a 
contemplated investment presented as a time series of dollar 
disbursements and receipts." [Ref. 15:p. 121]. Future funds 
are of less value to an organization than current equivalent 
funds, hence cash flows are discounted back to correspond to 
equivalent current funds. There are several methods for 
discounting cash flows once they been determined for each 
project. Two popular methods that deserve further description 
are present value and internal rate of return. 

a. Present Value 

Present value refers to the amount of funds 
necessary to invest now at a specified interest rate that is 
equivalent to the future cash disbursements and receipts of 
the project. In mathematical terms, the present value is the 
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reciprocal of future value and is calculated by dividing the 
future value by one plus the specified interest rate. Because 
a discounted cash flow is a function of both the interest rate 
and points of time in the future, it may be necessary to 
determine a series of cash flows for a project to reflect the 
passage of time before its completion. This is a simple 
summation process of the individual (usually yearly) present 
values of the cash flow. [Ref. 15:p. 129] The general 
equation for calculating present value is: 

P = F/(l+i) n 



where : 

/ 

P = Present value, 

F = Future cash flow amount, 

i = Set discount or interest rate 

(expressed as a decimal value, not a percentage) , 

n = Number of periods separating present and ture 
time periods. [Ref. 15 :p. 131] 

A slight modification to this model that 
frequently appears in the literature is net present value. 
Instead of calculating the cash flow and then determining its 
present value, one calculates the present value of monetary 
benefits and the present value of costs. Net present value is 
determined by subtracting the present value of costs from the 
present value of benefits. [Ref. 14 :p. 27] In both cases. 
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any projects with a negative present value would be 
eliminated . 

It is possible to use present value by itself as 
a screening tool but when ranking projects the additional step 
of developing a benefit/cost ratio allows the organization to 
incorporate a factor for the likelihood of technical success, 
b. Internal Rate of Return 

The internal rate of return is the discount rate 
that results in a present value of zero. This means that the 
effective rate of interest anticipated to be earned by the 
money invested in the project is equal to the present value of 
returns of existing projects. The usual method for 

determining the internal rate of return is trial and error. 
This process is naturally facilitated by the use of computers 
and iterative logic programming and most organizations use 
software packages rather then resort to the time=consuming 
process of graphic interpolation. [Ref. 15 :p. 135] 

Once the internal rate of return is known it is 
compared to the organization's minimum acceptable rate of 
return and accepted or rejected accordingly. 

2 . Advantages and Disadvantages 

The main benefit of the economic method is that it is 
a widely accepted, easily understood technique that adds a 
more quantitative approach to project selection than scoring. 
The capital budgeting techniques are well defined, and 
application procedures are found in both engineering economics 
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and in practice. [Ref. 6:p. 21] It is particularly appealing 
to non-research financially oriented managers. Economic 
methods are usually used to select product-oriented IR&D 
projects. 

There are several disadvantages associated with this 
method. It is designed to consider only financially-based 
resources and ignores such resources as space, manpower, etc. 
Because time plays a major role in the determination of 
present value, e onomic methods favor near-term ' r short range 
projects that provide incremental increases (possibility of 
technical success) to the present business. [Ref. 14 :p. 27] 
However, current discount methods properly applied are able to 
adjust for this shortcoming. 

The information to construct this model is widely 
dispersed throughout the organization. The R&D department 
provides information on the likely input mix, technology 
requirements, and resource overlaps; production managers 
provide cost estimates; the marketing department provides 
sales or user estimates; the finance department provides 
overall budget constraints; and top management provides the 
long range strategic guidance or value. The more dispersed 
information is throughout the organization the more difficult 
it is to obtain accurate dollar estimates. [Ref. 16:p. 30] 
Finally, even though the end result is a quantitative model, 
the inputs for the equations are still subjective estimates. 
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D. CONSTRAINED OPTIMIZATION METHODS 



The constrained optimization methods use linear and/or 
dynamic programming techniques in order to maximize benefits 
while selecting an optimal mix or portfolio of projects. 
These methods were developed because the process of project 
selection is not limited to selecting the best projects at a 
given point of time but rather is an on-going dynamic process 
of resource allocation between proposed and existing projects. 
The problem of resource allocation is created by the high 
probability that the requirements of the new proposed projects 
and existing projects will exceed current and forecasted 
capabilities. These models were developed to aid decision- 
makers in developing the optimal portfolio from a large number 
of feasible options. [Refs. 17:p. 119; 18:p. 36] 

A number of authors have developed models based on these 
assumptions. [Refs. 2:p. 127; 17:p. 119; 19:p. 28] One 
clearly-defined, easily understood model is the one published 
by A.C. Bell and A.W. Read, which extends the capital 
budgeting techniques of the economic method to accommodate 
additional features [Ref. 19:p. 27]. The features 
incorporated into this model include a variety of resource 
constraints, a sequence of future time periods, and 
alternative versions of projects. 

Alternative versions of the project can reflect such 
features as different rates of progress, alternative start 
periods, alternative technical approaches, etc. 
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Mathematically, the j ,h version of project i is expressed as the 
variable, x (j . The model is designed to produce a value for x,, 
of 1 or 0, where 1 indicates selection and 0 indicates 
rejection of the project. The formulation of the model is as 
follows : 



n m, 

maximize Z = £ £ h) (j x,, 

i=l j=l 



m, 

subject to: V x,. < 1 

j=l 



n 



m, 

.1 .1 

i=l j=l 



a i|kp X l| — ^Kp 



where : 

x,, = 0 or 1 , 

l. f 2 , m m • / 1*1 f 

j — 1/2, ...,111,, 

k =1,2, . . . , N, 

P = 1 / 2 , ...,P, 

b,, = Estimated value of version j of project i, 

m, = Number of alternative versions of project i, 

n = Number of projects, current and proposed, 

a ljKp = Amount of resource k planned for version j of 
project i in period p, 

= Overall availability of resource k in period p, 
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N = Number of resource categories, 

P = Number of planning periods. [Ref. 17: p. 120] 

This method requires careful analysis of the characteris- 
tics that form the parameters of the model. Perhaps the 
easiest parameter to define is time: the "planning horizon" 
or overall planning period, the number of planning periods and 
their length. Each resource must be clearly defined for each 
planning period by careful evaluation of current and fore- 
casted capacities. Examples of resources under consideration 
include money, manpower, facilities, supplies, etc. Estimated 
values of each version can be determined using the capital 
budgeting techniques discussed in the previous section. 

If the organization attempts to add more objectives and/or 
constraints, the model becomes too complex for linear 
programming. One solution is the use of dynamic programming, 
which decomposes a large mathematical model into a number of 
small problems that are solved recursively at each stage. 
[Ref. 20 :p. 721] 

1 . Advantages and Disadvantages 

This method is very attractive theoretically because 
it is designed to optimize quantitative measurements while 
meeting organizational constraints. Another advantage is its 
ability to incorporate many of the complex aspects of the 
selection process, such as existing projects, cross 
technology, manpower flexibility within the organization, 
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recruitment options, etc. Despite these advantages, this 
method is not widely used. [Refs. 2:p. 127; 17:p. 119] 

One of the major reasons for the limited use of this 
method is its relative complexity when compared to other 
available methods. It requires analysis of many if not all 
aspects of the organization; it attempts to optimize critical 
limited resources; it incorporates existing projects in its 
calculations; and it deals with a planning horizon to 
accommodate different decision periods. Another problem ith 
the model described, which is present in many optimization 
models, is that it assumes that once a project has been 
selected it will be completed. The model thus fails to 
account for future organizational decisions that occur over 
the life cycle of the project. Input data requirements for 
this method are difficult to obtain with any degree of 
precision or confidence. For example, the calculation of the 
time-cost trade-offs among the various resources is a very 
difficult task. [Ref. 18:p. 38] 

E. DECISION THEORY METHODS 

Decision theory methods focus on the trade-offs that exist 
among a group of projects over several periods of time. These 
methods involve the determination of alternative strategies 
available at different periods over the life of a project and 
an evaluation of the potential risk associated with each 
alternative. Normally, by means of simulation analysis, a 
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range of outcomes in the form of a probability density 
function are generated in order to define: (1) the expected 
value of the project, (2) the potential variance associated 
with that outcome, and (3) the project's probability of 
success. [Ref. 6:p. 22] A description of this decision 
process is illustrated by the construction of a decision tree 
for each project. 

A decision tree provides the framework to diagrammatical ly 
represent the multiple stages that a project undergoes over 
time and the uncertainties associated with those stages. In 
other words, it provides the means to evaluate the risks and 
trade-offs associated with the different alternative 
strategies. It is a means to display the complex and 
stochastic nature of R&D projects in a concise manner. 

Figure 1 illustrates the decision tree's ability to depict 
a number of characteristics. The tree extends over three time 
periods and incorporates two resource types. It illustrates 
a situation where the organization is faced with an initial 
decision point followed by an uncertain intermediate technical 
outcome. It not only illustrates technical outcomes but 
future decision points facing the organization. The advantage 
of this type of format is that it establishes a development 
path for each possible alternative, which clearly illustrates 
probability patterns and decision points for those 
alternatives on a pre-determined, common time scale. It is 
assumed that the discrete intervals of the time scale are fine 
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enough to assume that decisions are made at the beginning of 
the time periods. Simply stated, this method is capable of 
representing uncertainties in the duration, resource inputs 
and technical outcome at each stage for each alternative. 
[Ref. 17 : p . 120] 
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Figure 1. Decision Tree for Project i [Ref. 21:p. 951] 



The establishment of de ision trees for the projects under 
consideration will create a portfolio of opportunities. At 
this point, under the assumption of limited resources, it is 
necessary to determine which subset of projects should be 
pursued in order to optimize the overall objectives of the 
organization. 

One method is to maximize the weighted expected value of 
the final values of the projects pursued by the organization. 
The weight of the expected value is the probability associated 
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with that final state or the probability that the indicated 
state occurs. The following model illustrates this 
maximization process. 



n m, 

maximize Z = £ £ P(m ij ) b t( 

i=l j=i 



subject to only one j for each i 



P(m l( ) > 0 for all alternatives 



m, 

y P( m i|) = P(n\,) + P(m, 2 ) + . 
j 4 l 

for all projects 



+ POO = 1 



where : 

i 1 , 2 f . . . , n f 

J —1,2/ . . . , m, , 

b,, = Estimated value of version j of project i, 

Ptm^) = Probability of version j of project i, 
m, = Number of alternative versions of project i, 

n = Number of projects. [Ref. 20:p. 575] 



The above approach for selecting the optimal subset can 
develop into an exceedingly large problem. An alternative 
approach is based on the use of heuristics and simulation to 
generate a number of "good" solutions, leaving the final 
selection to the decision-maker. Another possibility is to 
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use heuristics to develop upper -^nd lower bounding solutions. 
[Ref. 21 :p. 953] 

1 . Advantages and Disadvantages 

Decision theory methods are useful in evaluating 
applied research projects, those projects that have well 
defined technical or commercial objectives. It provides a 
high degree of flexibility to demonstrate numerous possible 
versions of each proposed project. It attempts to address the 
complexity that exists in the real world. The above decision 
tree approach specifically handled the set of resources 
required over the life of the projects, possible technical 
outcomes, and estimated values of end states. [Ref. 17 :p. 
121 ] 

The primary problem associated with decision theory 
methods is determining the degree of detail for a given 
project and gathering the information needed to determine 
probability estimates for each state and the expected value of 
that state. Prior to the construction of the decision tree 
the following estimates need to be completed: (1) cost 
estimates for resources required for each alternative, (2) 
benefits associated with each alternative, and (3) the 
probability distribution for each outcome. The large amount 
of information required means that this is a costly method to 
implement and maintain. 
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III. NAVAL RDT&E CENTERS 



A. INTRODUCTION 

There are currently eight Naval Research, Development, 
Test and Evaluation (RDT&E) Centers operating under the 
guidance of the Naval Space and Warfare Systems Command [Ref. 
22]. Each of these RDT&E centers operates an Independent 
Research (IR) and Independent Exploratory Development (IED) 
program in their areas of expertise. The U.S. Navy divides 
its projects into these two categories rather than defining 
research and development as one unit. The IR portion refers 
to all efforts of scientific study and experimentation 
directed toward increasing fundamental knowledge and 
understanding in those fields of the physical, engineering, 
environmental, and life sciences related to long-term national 
security needs; the IED portion refers to all efforts, short 
of major development, directed toward the solutions of 
specific military problems [Ref. 23 :p. 3]. This chapter will 
discuss the basic IR/IED programs these centers manage and 
describe each center in terms of its mission, areas of 
expertise and scope of its IR/IED program. 

B. INDEPENDENT RESEARCH AND INDEPENDENT EXPLORATORY 

DEVELOPMENT PROGRAMS 

The IR/IED program is designed to promote scientific and 
technological growth in Naval RDT&E centers and the 
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development of knowledge and technology of interest to the 
Navy [Ref. 20 :p. l]. These programs are intended to support 
initial research and development in areas critical to the 
mission of each Navy center. Funding for IR/IED programs is 
furnished by the Office of Naval Research and the Office of 
Naval Technology in the form of discretionary funding under 
the control of the Technical Directors at each RDT&E center 
[Refs. 24:p. 1; 25:p. 3]. 

These programs enable individual scientists and 
technologists to conduct self-initiated research and 
development of promising but speculative solutions outside the 
requirements of normal funding authorization. This emphasis 
on innovation can provide an important and rapid test of 
promising new technology and fill gaps in a center's research 
and development program. [Ref. 26 :p. i] 

Even though procedural differences exist between the 
different RDT&E centers, the overall management philosophy is 
the same. An annual request for proposals is ir ued early in 
the year which contains preliminary guidance. After an 
initial review by IR and IED program managers, claimants are 
asked to make an oral presentation to specialist panels of 
qualified scientists and engineers. The evaluations by these 
panels are the primary tools used in the decision process by 
the IR and IED program managers to select projects. The 
program managers then prepare a suggested program which is 
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reviewed and approved by the Technical Director. [Ref. 27:pp. 
2-3] 



C. DAVID TAYLOR RESEARCH CENTER ( DTRC) 

1 . Mission 

The mission of DTRC is to be the principal Navy RDT&E 
center for Naval vehicles and logistics and for providing 
RDT&E support to the U.S. maritime administration and the 
maritime industry [Ref. 22]. 

2 . Expertise 

The DTRC conducts research and development in nine 
technical departments: (1) Ship Systems Integration, (2) Ship 

Hydromechanics, (3) Ship Structures and Protection, (4) Ship 
Acoustics, (5) Ship Electromagnetic Signatures, (6) Propulsion 
and Auxiliary Systems, (7) Ship Materials Engineering, (8) 
Aviation, and (9) Computation, Mathematics, and Logistics 
[Ref. 22]. Primary areas of concern are new vehicle concepts, 
ship and aircraft compatibility, ship trials and the 
development of vehicle technology. Other areas addressed 

include hull-form, structures, propulsion, silencing, 
maneuvering and control, auxiliary machinery, environmental 
effects, pollution abatement, logistics research, computer 
techniques and software for analysis and design. Some 
specific research concerns include such projects as high 
strength hulls; naval machinery in such areas as power 
systems, ship automation and control, machinery dynamics, 
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mechanical systems and components, electrical systems 
integration, and shipboard energy conservation; underwater 
acoustics and ship vibrations; and metals and alloys, 
corrosion, fuels and lubricants, paints, welding and 
fabrication. [Ref. 28:p. 186] 

3 . IR/IED Program 

During the fiscal year 1989, DTRC funded 25 IR 
projects at a cost of $2,528,000 and 18 IED projects at a cost 
of $1,505,000. Nine general IR areas were investigated. They 
were; (1) acoustics, (2) applied mathematics, (3) physical 
and mechanical metallurgy, (4) ceramics, glasses, and related 
inorganic solids, (5) hydromechanics, (6) aeromechanics, (7) 
structural mechanics, (8) electrical power generation, and (9) 
a miscellaneous category for in-house projects. The IED 
projects were all in one general area. Naval vehicles. [Ref. 
29 : pp . 95-99] 

D. NAVAL A IR DEVELOP 'ENT CENTER (NADC) 

1 . Mission 

The mission of NADC to be the principal Navy RDT&E 
center for Naval aircraft systems less aircraft-launched 
weapon systems [Ref. 22]. 

2 . Expertise 

The NADC is divided into three Warfare Systems 
Departments and four Engineering and Technology Departments; 
Antisubmarine Warfare Systems, Tactical Air Systems, Battle 
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Force Systems, Communication Navigation Technology, Air 
Vehicle and Crew Systems Technology, and Systems and Software 
Technology. Primary areas of effort are airborne ASW systems; 
airborne expendable microwave countermeasures; air command and 
control systems; airborne communications systems; air crew 
equipment and life support; airborne active and passive 
search, reconnaissance and surveillance systems and equipment; 
navigation systems, both inertial and autonomous, for air, 
surface and subsurface platforms; Naval airborne targets; 
Naval air vehicles including unmanned air vehicles; aircraft 
systems; and air vehicle modification and equipment 
installation. [Ref. 22] 

3 . IR/IED Program 

During the fiscal year 1989, NADC funded 25 IR 
projects at a cost of $2,871,000 and 16 IED projects at a cost 
of $1,633,000. Some examples of the types of IR projects 
funded include emission studies, diffusion modeling, high 
temperature superconductors, laser diode velocimetry, 
nonlinear model development, and fracture science. Examples 
of IED projects include ASW threat plan recognition, gate 
array technology for processors, advanced parallel processor 
development, laser radar returns, artificial intelligence for 
unmanned air vehicles and microwave communications. [Ref. 26: 
pp. 6. 1-7.2] 
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E. NAVAL COASTAL SYSTEMS CENTER (NCSC) 

1 . Mission 

The mission of NCSC is to be the principal Navy RDT&E 
center for mine and underseas countermeasures, special 
warfare, amphibious warfare, diving and other Naval missions 
that take place primarily in the coastal regions [Ref. 22]. 

2 . Expertise 

The program work at NCSC is divided into ten main 
areas of concern: (1) Airborne M- ne Countermeasures, (2) 

Surface Ship Mine Countermeasures, (3) Sonar and Torpedo 
Countermeasures, (4) Amphibious Warfare Support, (5) Marine 
Corps Tactical Deception, (6) Marine Corps Land Mine Warfare, 
(7) Coastal/Special Warfare Support, (8) Ocean Engineering and 
Mechanical Engineering, (9) Warfare Analysis, and (10) Advance 
Engineering and Technology Disciplines [Ref. 22]. 

3 . IR/IED Program 

During the fiscal year 1989, NCSC funded nine IR 
projects at a cost of $896,000 ind ten IED projects at a cost 
of $555,000. IR projects were in the areas of acoustic 
sensors, corrosion, electromagnetic detection, superconducting 
materials and devices. IED projects included such topics as 
multispectral imaging techniques, AUV decision management, 
copolymer hydrophones, curved lens development, and corrosion 
studies. [Ref. 30:pp. A.1-B.2] 
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F. NAVAL OCEAN SYSTEMS CENTER (NOSC) 

1 . Mission 

The mission of NOSC is to be the principal Navy RDT&E 
center for command, control and communications; ocean 
surveillance? surface- and air-launched undersea weapons 
systems; and submarine arctic warfare [Ref. 22]. 

2 . Expertise 

The program work at NOSC is divided into nine major 
areas: (1) Command, Control and Communications, (2) Ocean 

Surveillance, (3) Antisubmarine Warfare, (4) Ocean Sciences, 
(5) Ocean Engineering, (6) Submarine Arctic Warfare, (7) 
Intelligence Support Systems, (8) Biosystems Research, and (9) 
Support Technologies, including integrated circuit design and 
fabrication [Ref. 22]. A short list of examples of specific 
research areas include: automated integrated and laser 

communications systems; development of ship, shore and 
submarine communications systems; major surveillance systems, 
including autonomous arrays, towed arrays, deployable arrays, 
fixed distribution systems, active systems and systems for 
Arctic applications? manned and unmanned submersibles and 
underwater work and search systems; development of electro- 
optic devices for communications, surveillance, weapon 
delivery, and other military applications; underseas 
surveillance systems, including sonar signal processing, 
sensors, data acquisition systems, and transduction science; 
and manufacturing technology in materials, microwave tubes. 
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fiber optics, weapon systems, design and processing of 
integrated circuits, and robotics applications [Ref. 28 :p. 
185] . 

3 . IR/IED Program 

During the fiscal year 1989, 33 IR projects at a cost 
of $3,151,000 and 23 IED projects at a cost of $1,680,000 were 
funded. The call for IR proposals stressed the areas of 
Command, Control and Communications, ASW research, novel 
solid-state materials and devices, signal and image 
processing, secure survivable communications, and arctic 
research. The call for IED proposals emphasized the areas of 
artificial intelligence, arctic warfare, artificial neural 
networks, electro-optics, software engineering, and network 
technology. [Ref. 25 :p. 3] 

G. NAVY PERSONNEL RESEARCH AND DEVELOPMENT CENTER (NPRDC) 

1 . Mission 

The mission of NPRDC is to be the principal Navy RDT&E 
center for manpower, personnel, education, training, and human 
factors and for providing technical support to the Chief of 
Naval Operations in these areas [Ref. 22]. 

2 . Expertise 

The NPRDC is divided into six departments, which 
concentrate on the major areas of its mission: (1) Manpower 
Systems, (2) Personnel Systems, (3) Testing Systems, (4) 
Training Technology, (S' Training Systems, and (6) Human 
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Factors [Ref. 22]. Actual areas of research and development 
addressed by the NPRDC are management systems, personnel and 
occupational measurement, career development and retention, 
motivation and productivity, instructional technology, 
training systems, and command and support systems [Ref. 28 :p. 
186] . 

3 . IR/IED Program 

During the fiscal year 1989, NPRDC funded six IR 
projects costing $250,000 and four IED projects costing 
$195,000. IR projects were brain activity during visual 
recognition, diagrams for learning procedural tasks, memory 
performance, tutoring in technical training, stable 
performance of a complex cognitive task, and instructional 
analysis. IED projects were military recruitment quality, 
optimization of nonlinear objectives, personnel loss 
forecasting, and decomposition methods. [Ref. 24:pp. 2-3] 

H. NAVAL SURFACE WARFARE CENTER (NSWC) 

1. Mission 

The mission of NSWC is to be the principal Navy RDT&E 
center for surface ship weapons systems, ordnance, mines, and 
strategic systems support [Ref. 22]. 

2 . Expertise 

The program work at NSWC is divided into seven major 
areas: (1) Combat Systems, (2) Surface-launched Weapon 
Systems, (3) Underwater Weapon Systems, (4) Strategic Weapon 
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Systems, (5) Electromagnetic Combat, (6) Protection of Weapon 
Systems, and (7) Autonomous Weapon Systems [Ref. 22]. Some 
specific research areas being investigated include Nitinol- 
using devices; penetrameters or image quality indicator 
studies for radiographic efforts; thermal analysis of changes 
to physical or chemical properties of materials; acoustic 
testing of arrays, projectors/hydrophones, and special 
acoustic devices; hardening of digital electronics against 
gamma radiation; and water-entry . i water-exit phenomena 
[Ref. 28 :p. 123]. 

3 . IR/IED Program 

During the fiscal year 1989, NSWC funded 51 IR 
projects at a cost of $4,141,000 and 22 IED projects at a cost 
of $2,457,000. Some examples of the type of IR projects 
funded included accelerator-based atomic physics, nonlinear 
dynamics and fractals, high-temperature superconducting wire, 
neural network technology, munitions ch "istry, superlattices, 
biotechnology, ceramic science and pc ner science. Some 
examples of the type of IED projects funded included 
superconducting wire, underwater warhead technology thrust, 
neural networks, Ada for SIMD processors, expert systems, and 
water vapor absorption of radiation. [Ref. 27:pp. 12.3-12.11] 
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I. NAVAL UNDERWATER SYSTEMS CENTER (NUSC) 

1 . Mission 

The mission of NUSC is to be the principal Navy RDT&E 
center for submarine warfare and submarine weapon systems 
[Ref. 22]. 

2 . Expertise 

NUSC is organized into major product line directorates 
and departments. Those principal product lines are: 

Submarine Integrated Combat Systems, Submarine Sonar, Surface 
Ship Sonar, Submarine Electromagnetic Systems, Submarine 
Combat Control Systems, Torpedo Systems, Submarine Tactical 
Missile Systems, Launcher Systems, Underwater Target 

Simulators, Undersea Ranges Development and Operation, and 
Warfare Analysis. [Ref. 22] Areas of unique expertise 

include acoustic arrays, signal processing, acoustic 

transducers, modeling and analysis, environmental acoustics, 
hydrodynamics, and propulsion [Ref. 28:p. 186]. 

3 . IR/IED Program 

During the 1989 fiscal year IR/IED funding totaled 
$4,958,000: $3,138,000 was used to fund 32 research projects 

and the remaining $1,820,000 funded 19 exploratory development 
projects. Fourteen technology areas were addressed by the 
projects funded. The 11 research areas investigated were 
Acoustics, Applied Mathematics, Arctic Research, Electrical 
Power Generation, Electromagnetic Wave Propagation and 
Radiation, Engineering Psychology, Hydrodynamics, Information 
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Processing Devices, Mathematical Statistics and Engineering 
Applications, Oceanography, and Structural Mechanics. The IED 
portion reported three areas of investigation: Support 

Technology, Target Surveillance and Weaponry. [Ref. 31:p. 

I. 3] 

J. NAVAL WEAPONS CENTER (NWC) 

1 . Mission 

The mission of NWC is to be the principal Navy RDT&E 
center for air warfare systems (except r.tisubmarine warfare 
systems) and missile weapon systems and the national 
range/facility for parachute test and evaluation [Ref. 22]. 

2 . Expertise 

Program work is directed toward air warfare systems, 
missile weapon systems, ordnance, foreign material 
exploitation, and support systems [Ref. 22]. Specific 
research areas include electronic circuits and systems, 
electronic quality assurance, microwave tube design, corrosion 
from salt water and weather effects, p_. otechnic chemistry, 
and testing of primary and secondary batteries [Ref. 28 :p. 
186] 

3 . IR/IED Program 

During the fiscal year 1989, NWC was allocated 
$4,042,000 for its IR program and $1,865,000 for its IED 
program [Ref. 28:p. 8]. One notable project was on the 



36 



development of an INTEL artificial neural network 
microelectronics chip [Ref. 32 :p. 12]. 

K. SUMMARY 

The information about the above funding levels came from 
the IR/IED Annual Reports issued by each center. The overall 
funding level was almost $34 million dollars, a relatively 
small amount considering the importance of the program. This 
program is the spark of creativity used to attract and retain 
highly qualified scientists and engineers in many diverse 
fields of study. Its importance to the strength and success 
of a facility needs to be periodically stressed. 

The wide variety of topics selected for investigation 
reflects the wide range of needs of today's U.S. Navy. Each 
center has to select and reject projects (hundreds of projects 
are rejected each year) that cover several major fields of 
study. The development of a fair and consistent selection 
process is an important part of this program. 
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IV. AN INTEGRATED APPROACH TO IR&D PROJECT SELECTION 



A. INTRODUCTION 

As demonstrated by Chapter III, Navy laboratories perform 
independent research and development in a wide range of 
fields. Responsibilities of Independent Research and 
Independent Exploratory Development (IR/IED) directors will 
increase even more once the pli. ned consolidation of the 
administrative and support services of the laboratories occurs 
and they are responsible for additional fields of study at 
more facilities. This is a period of change for research 
departments and many procedures are being evaluated, including 
the selection process of IR/IED projects. 

The four theoretical methods discussed in Chapter II each 
have distinctive advantages; it would be impossible for any 
one selection process to incorporate all of these advantages. 
One key characteristic of the project:; that assists i: 
determining which selection processes wou^d be appropriate is 
the degree of uncertainty associated with performing an 
accurate cost-benefit analysis on the project. By separating 
projects into categories determined by the degree of 
uncertainty associated with benefits and costs estimates, it 
is possible to tailor the selection process to a particular 
category and compare projects with similar characteristics. 
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The Navy has established a clear distinction between 
independent research and independent exploratory development, 
and the selection process for these two areas is performed 
separately. One advantage of establishing separate categories 
is that projects are grouped by similar characteristics prior 
to selection. It also permits the existence of two different 
selection procedures in order to take advantage of strengths 
of the different methods. 

B. INDEPENDENT RESEARCH PROJECTS 

Independent research projects can be characterized as the 
initial stage of a project, during which time the degree of 
uncertainty surrounding the project is high and the ability to 
perform a rigorous cost-benefit analysis is poor. 

1 . Scoring Implications 

Use of a scoring model in order to screen projects is 
an appropriate method to assess IR projects. However, a 
scoring model can be successful only if its criteria are based 
on organizational goals. Clearly defined criteria form the 
basis of communication between top management, expert 
evaluators and researchers. Chapter II illustrated how to set 
up the mechanics of a scoring model; it does not discuss the 
criteria against which a research project is judged. Any 
recommendation regarding the use of a scoring model must 
include the definition of key criteria. The approach 
developed by M.J. Cooper [Ref. 33] was designed to produce 



39 



well-defined parameters and provides a common language to 
discuss selection criteria. A modification of his approach 
for a military laboratory forms the basis for discussing key 
criteria. The three primary criteria examined are impact, 
feasibility and research merit. [Ref. 29:p. 29] 

2 . Criteria 

a . Impact 

Impact is the term used by M.J. Cooper to describe 
the utility of the project and can r simply defined as the 
reason for undertaking the project: Why should this project 
be considered? In seeking to answer this question the 
researcher needs to identify: (1) how his project meets a 
military service goal, and (2) the users who will benefit from 
this new technology. Association with a service goal shows 
that the project explores a technology with an actual utility 
to the military service. The identification of users 
establishes a poten^'al market for this new technology. 

One way c explore the potential benefits of the 
project is to define it in terms of the recognized and direct 
segment of the market affected: Does this project have 
applications for the Department of Defense; the U.S. Navy; a 
community within the U.S. Navy (surface, submarine, aviation, 
etc.)? A project that provides limited support to a critical 
national goal can be of greater importance than one that is of 
major importance to a small interest group. This means that 
a project's market should be defined in two stages. The first 



40 



stage is to define the importance of the goal it seeks to 
support. The second stage is to identify the level of 
importance of the project in meeting that goal: To what 

degree is the next stage of the goal reliant on the successful 
completion of this project? The following scale in descending 
order of importance of organizational goals, established by M. 
J. Cooper, has been slightly modified for a Navy organization: 

- Recognized National Goal. 

Recognized Department of Defense Goal. 

- Recognized Navy Goal. 

- Supports Navy Goal. 

- General Research Function of Laboratory. 

- Limited Interest to Select Subgroup. [Ref. 29 :p. 30] 

The identification of end users means that, in 
addition to the establishment of a potential market, the 
researcher is able to communicate with those users. The 

interaction between the researcher and users can have a major 
impact on the final success of the project, especially 
concerning modifications of the project and its impact on 
users' procedures. For example, if the project has 
applications to another established research project, 
coordination between the two research groups can ease the 
incorporation of successful applications into the established 
program. By establishing communications with the end users 
and incorporating their concerns, the research group is able 
to mitigate the users' resistance to change and develop a 
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commitment in the users to the project's success. The end 
users' attitudes toward the project can affect the ultimate 
acceptance of new procedures and products that result from the 
research. The reaction of the end users to the final product 
of the project can be rated according to the following scale: 
Direct User Participation. 

Substitution in Existing Market. 

Uncertainty, Indifference. 

- Conflicts, Displacements of Existing Procedures. [Ref. 

29 :p. 31] 

Under this system, impact is measured by a 
project's programmatic utility, which is discounted by the 
effectiveness of the interaction with end users, 
b. Feasibility 

Feasibility is simply another term for risk or 
the likelihood of accomplishing the desired task. It is the 
probability that the project will achieve technical success 
given its planned time and resource constraints. [Ref. 8:p. 
223] Specific factors that influence possible success include 
technological risk, technological competence of the researcher 
and the availability of competent management personnel. 

Technological risk refers to the availability of 
technology necessary to complete the project. Research that 
is based on a mature technology or seeking to initiate an 
incremental change to an existing technology will naturally 
have less risk associated with it, than research that is 
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predicated on information not yet available. The following 
scale reflects the general level of knowledge in the field of 
study under investigation: 

Demonstrated Practice. 

Evidence Technology Exists. 

Uncertainties, but Supposedly Resolvable. 

Major Advances Required. [Ref. 29 :p. 31] 

Resource availability is a key factor in 
determining a project's feasibility. For this paper, 
technological competence refers not only to the qualifications 
of the primary investigator but also to the presence of 
appropriate technical skills and facilities needed by the 
research group to accomplish its task. One key resource which 
is not always included in the decision-making process is the 
availability of management personnel capable of supplying 
needed support services. The following scale reflects various 
levels of availability: 

Necessary Personnel, Skills, Facilities. 

Personnel, Skills, Facilities Generally Available. 

Requires Learning, Attaining New Skills (or facilities) . 

Demands Significant Effort to Acquire Facilities or 
Skills or Learn Skills. [Ref. 29:p. 31] 

Analyzing the resources needed to successfully 

complete the project provides the initial baseline for 

calculating the costs of a project. Determining a cost 
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baseline is the first step to establishing a monitoring 
program to be refined over the life of the project, 
c. Intrinsic Scientific Merit 

The implicit goal of a research organization is to 
contribute to the level of knowledge of science and 
technology. The long-term viability of the organization is 
best maintained by the sponsoring of high quality work. The 
ability to attract and keep the best personnel is a function 
of the type of research conducted by that organization. 
Another benefit of the presence of leading researchers is that 
other experts and institutions seek to exchange information 
with the organization's personnel, thus exposing personnel to 
the broadest outside knowledge. 

Scientific merit refers to the project's potential 
to contribute to new understanding of the phenomena being 
investigated. It provides the means for management to 
acknowledge the value of scientific and technological 
advancements. The following scale provides the means to rate 
research opportunity: 

Major Opportunity. 

Broad Impact. 

Complements Other Research Programs. 

Specific Technological Fix. [Ref. 29:p. 33] 

3 . Summary 

M.J. Cooper's model provides a common language for 
communication between researchers and management. It 
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establishes basic scales for ranking the different projects. 
It does not address the weights assigned to criteria. A study 
conducted at an Air Force research and development laboratory 
showed that service need and scientific merit accounted for 84 
percent of the explainable variance associated with project 
selection of the six criteria used. [Ref. 8:p. 226] The 
criteria studied were similar to the ones discussed above. 
While this may indicate one possible direction, the setting of 
weights is the responsibility of top management. Independent 
research and development funds are among the last that are 
controlled by the laboratories, which gives senior management 
the flexibility and opportunity to set priorities on the 
laboratory level. 

C. INDEPENDENT EXPLORATORY DEVELOPMENT 

Independent exploratory development projects range from 
fundamental applied research to breadboard hardware. The 
projects under consideration address specific problems and are 
usually more narrowly defined than IR projects. 

1 . Scoring Implications 

The information needed to conduct cost-benefit 
analysis on IED projects is obtainable using standard cost 
estimation techniques. Engineering economics is one such 
technique [Ref. 6:p. 21]. While this is an expensive method 
of calculating costs it is possible considering the types of 
projects undertaken. A review of annual reports of the 
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laboratories shows that projects are specialized and clearly 
defined [Refs. 24? 25? 26? 27? 29? 30? 31]. They are headed 
by experts with a comprehensive knowledge of the project's 
function and components, and supported by experienced cost 
estimators. The growing costs of research combined with 
increased government concerns regarding cost management favors 
the use of economic models of selection whenever possible. 
The introduction of such practices as zero-based budgeting 
techniques is another indication that once the information 
exists to assess risks and assign costs, economic-based models 
are appropriate. 

The simple linear economic model outlined in Chapter 
II is insufficient to be effective in practice. It assumes 
the use of dynamic programming to calculate the cost and 
benefit figures used to maximize the value of benefits. 
Minimally, the cost figure for the project is comprised of the 
cost of supplying the resources required to successfully 
complete the project and would be calculated as a smaller 
early problem. The same is true for benefits, which are 
calculated based on the potential market or uses for the 
project. 

The total cost figure has to account not only for the 
overall budget but is subject to resource constraints. Scarce 
resources need to be identified prior to considering project 
proposals, and if it is obvious that there is a serious 
scarcity issue, a constrained optimization model should be 
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should be used. The fact that requests for proposals are an 
annual event would indicate that a slight modification of the 
model discussed in Chapter II would be appropriate; namely, 
only one time period would be taken under consideration. 

If the projects are so clearly or narrowly defined 
that only one version of the project is under consideration, 
then the basic economic model is sufficient. However, the 
existence of alternatives is a distinct possibility. Even 
though the other models, constrained optimization and decision 
theory, propose means for handling alternatives, both methods 
are very expensive, and differences among alternatives may not 
justify the cost of a complete analysis of each alternative. 
It may be sufficient for the researcher to identify the 
preferred or most likely alternative and prepare a cost- 
benefit analysis on that alternative, rather than do a 
rigorous analysis on all possible alternatives. 

2 . Summary 

In situations where the information is available, 
the use of cost-benefit analysis provides an uniform method of 
comparing projects which is familiar to most managers. It 
also provides a solid basis for cost management of the 
project. Another advantage of the economic model is that it 
more clearly identifies needed resources than a scoring model 
and during a period of shrinking resources this issue has 
become more important. 
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C. MANAGEMENT ISSUES 



The selection of a project is only the beginning of the 
management of a project over its lifetime. As the project 
progresses, it is essential to periodically evaluate costs and 
benefits as uncertainty decreases. The ability to perform 
effective cost-benefit analysis is not only an efficient way 
to maximize benefits but also is an effective means of 
monitoring the program over its lifetime. The information 
provided during the selection process is the baseline from 
which a project is evaluated. 

N.R. Baker, A.S. Bean and S.G. Green studied 211 R&D 
projects; their findings indicate that the resolution of 
uncertainty is strongly related to the project's eventual 
success [Ref. 30:p. 29]. This is not restricted to financial 
considerations. They also demonstrated that while initial 
goal uncertainty does not significantly impact the project's 
final success or failure, the existence of clearly defined 
technological goals are statistically significant late in the 
life of a project. [Ref. 30:p. 32] This indicates that as a 
project progresses it should be reviewed, not only for costs, 
but its goals should be refined and clarified. One way to 
minimize failure is to establish a management program that 
focuses on reducing uncertainty over the course of a project. 
This indicates a formal periodic evaluation program would be 
useful which involves not only the research group but the 
financial department, end users and management. The research 
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group will have the primary responsibility for ensuring needed 
participation by end users and this should remain an informal 
relationship until formal testing begins. The manager 
assigned to the research group is responsible for maintaining 
open communications with the group in order to periodically 
chart progress, clarify uses, identify existing programs that 
can benefit from this project and initiate the transfer of the 
project to an existing program when appropriate, monitor 
resource requirements and provide liaison with financial 
department regarding costs and resource availability. 
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V. FUTURE ISSUES 



A. INTRODUCTION 

There are three issues facing the Navy ' s Research and 
Development program that have the potential to significantly 
affect the Independent Research and Independent Exploratory 
Development (IR/IED) program. These issues are: 

- The centralization and streamlining of Navy-wide 
Research, Development, Testing and Evaluation (RDT&E) 
responsibilities . 

The age of Navy laboratories, facilities and equipment. 

The current trend to focus on solving today's engineering 
problems rather than stressing development needed to meet 
tomorrow's challenges. [Ref. l:pp. 192-195] 

This chapter will discuss each of these issues from the 
perspective of how they impact the Navy's IR/IED program. 

B. CENTRALIZATION OF RDT&E RESPONSIBILITIES 

The main purpose of this program is to consolidate the 
management and support responsibilities of the numerous Navy 
facilities to five centers [Ref. 35] . This reorganization 
comes out of past Defense Management Reviews that called for 
the elimination of duplication of laboratory research, both 
within and across military services [Ref 36:p. 68]. This 

means that fewer IR/IED directors will be responsible for 
administering a larger program in terms of areas of 
investigation, budget and number of facilities [Ref. 35]. 
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This situation will impact on the selection process of 
IR/IED projects by making the administration of the selection 
process more complicated. At the present time, for every one 
proposal approved, approximately three proposals are submitted 
[Ref. 25:p. 3]. If the current trend noted in several of the 
annual reports prepared by the RDT&E centers continue then the 
number of proposals submitted annually will increase. [Refs. 
25; 27; 29] With the scientists and engineers who initiate 
the proposals dispersed over a greater geographical distance 
and investigating a greater variety of topics, the logistics 
involved in organizing the oral briefs and coordination of the 
expert panels will become more complex. This is an 
administrative problem that can be resolved. However, the 
final decision-makers are still the Technical Directors based 
on recommendations from the IR/IED directors. A problem that 
could develop, because IR/IED directors are not familiar with 
the additional functional areas, is that they may favor 
projects in areas where they are most knowledgeable. During 
this transitional phase a conscious effort must be made not 
to let traditional loyalties influence the selection process. 

Another, less obvious issue is the lessening of overlap 
research in the move to eliminate duplication [Ref. 36:p. 70]. 
Duplication is the wasteful repetition of efforts, while 
overlap is the intentional investigation of different 
technical approaches to a given problem. A significant 
portion of the IR/IED program is the investigation of risky 
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solutions to problems and can be classified as overlap 
research. A change in policy to eliminate overlap projects 
could have a major impact on the type of IR/IED projects 
selected. If risky alternative solutions are no longer being 
investigated then the risk of failing to find the best 
technical solution will increase. By not investigating 
overlap alternatives the military decreases its ability to 
deliver the highest quality solution to the problem. Another 
reason to pursue high-risk solutions to problems is that they 
often have high-payoffs in terms of technological quality and 
the discovery of additional applications not originally 
anticipated. 

C. AGE OF FACILITIES 

Navy RDT&E Centers' facilities and equipment are steadily 
becoming obsolete, inefficient and, in some cases, 
deteriorating. Currently over 73 percent of the Navy's 
permanent buildings used for research and development were 
constructed prior to 1960. Many of these facilities are of 
World War II construction. And the present renovation and 
replacement plan indicates that this is a worsening situation. 
[Ref. lsp. 194] 

Because the IR/IED program focuses on future technology, 
many of the projects require the use of state-of-the-art 
facilities and equipment. Resource constraint issues in the 
past have been the overall budget and, in some rare cases, the 
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termination of approved projects because a principal 
investigator was not available [Ref. 27:p. 258]. The lack of 
modern facilities and equipment and increased competition for 
those limited renovated and new facilities will corresponding- 
ly increase the importance of resource allocation as part of 
the selection process of IR/IED projects. This could result 
in downgrading the importance of scientific merit as a 
decision criteron and increasing the importance of economic 
factors, such as resource allocation. 

D. DEVELOPMENT PRIORITIES 

There exists a trend in the research and development 
programs run by the government laboratories to fund projects 
to find engineering solutions to current problems rather than 
focusing on developing the technology of the future. Superior 
technology has long been a part of U.S. strategy. An integral 
part of maintaining technological superiority is an active and 
productive IR/IED program with its long-term view and stress 
on new technologies. In conversations with IR/IED directors, 
it was mentioned that there has been a slight erosion in the 
IR/IED budget at some RDT&E Centers and concern was expressed 
about a possible trend [Refs. 37; 38]. Any significant 
decrease in the IR/IED budget would have a major impact on the 
technological edge enjoyed by the U.S. 
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E. SUMMARY 

The relatively small size of the Navy's IR/IED budget 
compared to the benefits it generates is the strongest 
possible argument for its continuation. This program supports 
technological investigations into high-risk and high-payoff 
areas of interest to the Navy. It enhances the recruitment 
and retention of top quality scientists and engineers into the 
Navy laboratory system. It develops the technological base in 
new and emerging technologies while building the expertise of 
personnel. It encourages the interaction between laboratory 
personnel, academia and industry. And it enhances the 
scientific reputation of Navy facilities and personnel. 

These advantages do not protect the program from general 
budget cuts or the facility and equipment constraints 
discussed in the previous sections. The need to optimize the 
available funds and resources is an important issue, and an 
effective selection process is one way to optimize the 
benefits of the program. As IR/IED directors become 
responsible for larger and more varied programs with fewer 
resources, the need to clarify goals and criteria is needed to 
ensure that the diversity of projects sponsored by the 
Navywide program does not suffer. A clearly-defined, well- 
planned selection process will encourage proposals and help 
maintain the flexibility of the current program. 
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